Faster Training of Structural SVMs with Diverse M-Best Cutting-Planes
نویسندگان
چکیده
Training of Structural SVMs involves solving a large Quadratic Program (QP). One popular method for solving this optimization problem is a cutting-plane approach, where the most violated constraint is iteratively added to a working-set of constraints. Unfortunately, training models with a large number of parameters remains a time consuming process. This paper shows that significant computational savings can be achieved by generating and adding multiple highly violated constraints at every iteration of the training algorithm. We show that generation of such diverse M-Best cutting-planes involves extracting diverse M-Best solutions from the loss-augmented score function. Our experiments on image segmentation and protein side-chain prediction show that the proposed approach can lead to significant computational savings, e.g., > 60% reduction in the number of training iterations. Finally, our results suggest that even greater savings are possible for more expressive models involving a larger number of parameters.
منابع مشابه
DivMCuts: Faster Training of Structural SVMs with Diverse M-Best Cutting-Planes
Training of Structural SVMs involves solving a large Quadratic Program (QP). One popular method for solving this QP is a cutting-plane approach, where the most violated constraint is iteratively added to a working-set of constraints. Unfortunately, training models with a large number of parameters remains a time consuming process. This paper shows that significant computational savings can be a...
متن کاملSupplementary Material for “DivMCuts: Faster Training of Structural SVMs with Diverse M-Best Cutting-Planes”
This section reviews the notation used in the main paper and revisits cutting-plane methods for training structured-output predictors. Notation. For any positive integer n we use [n] as shorthand for the set {1, 2, . . . , n}. We use y for a structured-output, and Y = (y1, . . . ,y|Y|) for a tuple of structured-outputs. Given a training dataset of input-output pairs {(xi,yi) |xi ∈X ,yi ∈Y}, we ...
متن کاملA Sequential Dual Method for Structural SVMs
In many real world prediction problems the output is a structured object like a sequence or a tree or a graph. Such problems range from natural language processing to computational biology or computer vision and have been tackled using algorithms, referred to as structured output learning algorithms. We consider the problem of structured classification. In the last few years, large margin class...
متن کاملTraining SVMs Without Offset
We develop, analyze, and test a training algorithm for support vector machine classifiers without offset. Key features of this algorithm are a new, statistically motivated stopping criterion, new warm start options, and a set of inexpensive working set selection strategies that significantly reduce the number of iterations. For these working set strategies, we establish convergence rates that, ...
متن کاملA Distributionally-robust Approach for Finding Support Vector Machines
The classical SVM is an optimization problem minimizing the hinge losses of mis-classified samples with the regularization term. When the sample size is small or data has noise, it is possible that the classifier obtained with training data may not generalize well to population, since the samples may not accurately represent the true population distribution. We propose a distributionally-robust...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2012